What is claimed is: 

1 . A method of registering a 2D (two-dimensional) x-ray image of a target with previously 
generated 3D scan data of said target, said x-ray image being characterized by an image plane 
defined by mutually orthogonal x- and y- coordinates, the method comprising: 

A. generating at least one reconstructed image from said 3D scan data; and 

B. determining the value of in-plane transformation parameters (x, y, 9) and out-of-plane 
rotational parameters (r, (p) for registering said reconstructed image onto said x-ray image, said 
parameters representing the difference in the position of the target as shown in said x-ray image 
as compared to the position of the target as shown by said image reconstructed from said 3D 
scan data; 

wherein r and cp represent the rotations of said target about first and second mutually orthogonal 

axes, said rotations being out-of-plane with respect to said image plane, said out-of-plane 

rotations representing the projection of said target onto said image plane; 

wherein x and y represent the amount of translation of said target within said image plane in the 

directions of said x- and y- axes, respectively, and 9 represents the amount of rotation of said 

target within said image plane about an axis perpendicular to both said x- and said y- axes; 

and 

wherein step B comprises: 

a. obtaining an initial estimate for said in-plane transformation parameters (x, y, 9) by 
multi-level matching in 3D (three dimensions), between said x-ray image and said reconstructed 
image; 

b. based on said parameters (x, y, 9) estimated in step a, performing an initial search in one 
dimension for each of said pair of out-of-plane rotation parameters (r, cp); and 

c. iteratively refining said in-plane parameters (x, y, 9) and said out-of-plane parameters (r, 
(p), until said parameters converge to a desired accuracy. 
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2. A method in accordance with claim 1 , wherein said 3D multi-level matching is performed 
sequentially in each of a succession of a plurality of image resolution levels, starting at the 
lowest resolution level and ending at the highest resolution level. 

3. A method in accordance with claim I , further wherein said 2D x-ray image of said target 
is obtained by transmitting through said target an imaging beam having a known position and 
angle relative to said target, and wherein said reconstructed image is a 2D synthesized DRR 
(digitally reconstructed radiographs) representing the radiographic image of said target that 
would be obtained with said imaging beam at said known position and angle, if said target were 
positioned in accordance with said 3D scan data. 

4. A method in accordance with claim 1, further comprising the steps of 

A. determining a plurality H and N(p of out-of-plane rotation angles, respectively, for said 
rotational parameters (r, cp); 

B. generating a plurality Nr* N(p of 2D reference images, one reference image for each of 
said plurality Nr and N(p of said out-of-plane rotation angles. 

5. A method in accordance with claim 1, further comprising the step of generating offline, 
before step a, a plurality of in-plane rotated 2D reference images, by performing a series of in- 
plane rotations on said reconstructed image. 

6. A method in accordance with claim 5, wherein said 3D matching process in step a is 
performed upon said in-plane rotated 2D reference images. 

7. A method in accordance with claim 1, wherein said 3D matching process in step a is 
performed using a similarity measure method. 

8. A method in accordance with claim 7, wherein said similarity measure method is based 
on a sum of absolute differences. 

1 8 
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9. A method in accordance with claim 1, wherein step c of iteratively refining said in-plane 
and out-of-plane parameters comprises: 

d. refining the in-plane translation parameters (x, y), to increase the accuracy of said 
parameters; 

e. refining the in-plane rotation parameter (6) based on said out-of-plane rotation parameters 
(r, (p) searched in step b, and on said refined in-plane transformation parameters (x, y) fi"om step 
d; 

f separately refining each of the out-of-plane rotation parameters (r, (p), based on said 
refined in-plane translation parameters fi'om step d, and said refined rotation parameter fi-om step 
e; 

g. iteratively and sequentially repeating steps d, e, and f, until a predetermined accuracy is 
reached; and 

h. refining once more said out-of-plane rotation parameters (r, cp). 

10. A method in accordance with claim 9, wherein step d of initially refining the in-plane 
translation parameters is performed by sub-pixel matching in two dimensions. 

11. A method in accordance with claim 9, wherein step e of refining the in-plane rotation 
parameters is performed by ID (one dimensional) interpolation. 

12. A method in accordance with claim 9, wherein step f of separately refining said out-of- 
plane rotation parameters is performed through a ID (one dimensional) search. 

13. A method in accordance with claim 9, wherein step h of refining said out-of-plane 
rotation parameters (r, cp) is performed by ID interpolation. 

14. A method in accordance with claim 1, wherein said predetermined accuracy is sufficient 
to achieve a resolution of less than about Inun. 
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15. A method in accordance with claim 1, wherein said 3D scan data comprise at least one of 
CT scan data, MRI scan data, and PET (positron emission tomography) data, 

16. A method in accordance with claim 1, wherein said ID search for said out-of-plane 
rotation parameters in step b is performed using a similarity measure. 

17. A method in accordance with claim 16, wherein said similarity measure is based on 
pattem intensity. 

18. A method in accordance with claim 1, wherein the search space for said ID search in step 
B is the full search range of out-of-plane rotation angles, and said full search range is sampled by 
one degree increments. 

19. A method in accordance with claim 9, wherein steps d, e, and fare performed using a 
similarity measure based on pattem intensity. 

20. A method in accordance with claim 1, further comprising the step of processing said 2D 
x-ray image, after step A and before step B, so as to match the orientation, image size, and bit 
depth of said x-ray image with the orientation, image size, and bit depth of said reconstructed 2D 
image. 

21. A system for registering at least one 2D radiographic image of a target with at least one 
image reconstructed from previously generated 3D scan data of said target, said radiographic 
image being characterized by an image plane defined by mutually orthogonal x- and y- axes, the 
system comprising: 

a. means for providing said 3D scan data of said target; 

b. a radiation source for generating at least one radiographic imaging beam having a known 
intensity, and having a known position and angle relative to said target; 
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c. an imaging system for generating a 2D radiographic image of said target in near real 
time; and 

d. a controller, including: 

i) means for generating said at least one reconstructed 2D image of said target, using 
said 3D scan data, and using said known location, angle, and intensity of said imaging beam; and 

ii) software for determining a set of in-plane transformation parameters (x, y, 9) and 
out-of-plane rotational parameters (r, <p), said parameters representing the difference in the 
position of the target as shown in said x-ray image as compared to the position of the target as 
shown by said 2D reconstructed images; 

wherein r and (p represent the rotations of said target about first and second mutually orthogonal 
axes, said rotations being out-of-plane with respect to said image plane, said out-of-plane 
rotations representing the projection of said target onto said image plane; and 
wherein x and y represent the amount of translation of said target within said image plane in the 
directions of said x- and y- axes, respectively, and 9 represents the amount of rotation of said 
target within said image plane about an axis perpendicular to both said x- and said y- axes. 

22. A system in accordance with claim 21, wherein said software for determining said in- 
plane and out-of-plane rotational parameters comprises: 

means for performing a 3D multi-level matching to determine an initial estimate for said in-plane 
transformation parameters (x, y, 9); 

means for performing a ID search for each of said pair of out-of-plane rotation parameters (r, (p) 
based on said initially estimated in-plane parameters (x, y, 9), and 

means for iteratively refining said in-plane parameters (x, y, 9) and said out-of-plane parameters 
(r, cp), until a desired accuracy is reached. 

23. A system in accordance with claim 21, wherein said radiation source comprises an x-ray 
source, said 2D radiographic image comprises a 2D x-ray image, and said reconstructed image 
comprises a 2D DRR. 
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24. A system in accordance with claim 21 , wherein said controller further comprises: 

A. means for determining a plurality H and N^p of out-of-plane rotation angles, respectively, 
for said rotational parameters (r, (p); and 

B. means for generatmg a plurality H* Nq, of 2D reference images, one reference image for 
each of said plurality Nr and N^p of said out-of-plane rotation angles. 

25. A system in accordance with claim 21 , wherein said controller further comprises means 
for generating offline a plurality of in-plane rotated 2D reference images by performing a series 
of in-plane rotations on said reconstructed image. 

26. A system in accordance with claim 22, wherein said 3D multi-level matching means 
performs sequentially in each of a succession of a plurality of resolution levels, starting at the 
lowest resolution level and ending at the highest resolution level. 

27. A system in accordance with claim 22, wherein said 3D multi-level matching means 
comprises similarity measure means based on a sum of absolute differences. 

28. A system in accordance with claim 22, wherein said means for iteratively refining said 
in-plane and out-of-plane parameters comprises: 

d. means for refining the in-plane translation parameters (x, y), to increase the accuracy of 
said parameters; 

e. means for refining the in-plane rotation parameter (9) based on said out-of-plane rotation 
parameters (r, (p) searched in step b, and on said refined in-plane transformation parameters (x, y) 
firom step d; 

f means for separately refining each of the out-of-plane rotation parameters (r, (p), based on 
said refined in-plane translation parameters firom step d, and said refined rotation parameter firom 
step e; and 

g. means for iteratively and sequentially repeating steps d, e, and f, until a predetermined 
accuracy is reached, and for refining once 

22 

BST99 1366487-1.063564.0060 



more said out-of-plane rotation parameters (r, cp). 

29. A system in accordance with claim 22, wherein said means for refining the in-plane 
translation parameters comprises 2D sub-pixel matching means. 

30. A system in accordance with claim 22, wherein said means for refining the in-plane 
rotation parameters comprises ID (one dimensional) interpolation means. 

31. A system in accordance with claim 22, wherein said means for separately refining said 
out-of-plane rotation parameters comprises means for performing one or more ID searches. 

32. A system in accordance with claim 22, wherein said means for refining said out-of-plane 
rotation parameters (r, (p) comprises ID interpolation means. 

33. A system in accordance with claim 22, wherein said desired accuracy is sufficient to 
achieve a resolution of less than about 1mm. 

34. A system in accordance with claim 21, wherein said 3D scan data comprise at least one 
of CT scan data, MRI scan data, and PET (positron emission tomography) data. 

35. A system in accordance with claim 21, wherein said means for performing a ID search 
for said out-of-plane rotation parameters comprises means for performing a similarity measure 
based on pattem intensity. 

36. A system in accordance with claim 22, wherein said means for refining the in-plane 
translation parameters (x,y), said means for refining the in-plane rotation parameter (9), 
and said means for separately refining said out-of-plane rotation parameters (r, cp) comprises 
means for performing one or more similarity measures based on pattem intensity. 
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37. A system in accordance with claim 22, further comprising means for processing said 2D 
x-ray image so as to match the orientation, image size, and bit depth of said x-ray image with the 
orientation, image size, and bit depth of said reconstructed image. 
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